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Abstract 

In this paper we introduce a variant of pushdown dimension called bounded push- 
down (BPD) dimension, that measures the density of information contained in a 
sequence, relative to a BPD automata, i.e. a finite state machine equipped with an 
extra infinite memory stack, with the additional requirement that every input symbol 
only allows a bounded number of stack movements. BPD automata are a natural 
real-time restriction of pushdown automata. We show that BPD dimension is a ro- 
bust notion by giving an equivalent characterization of BPD dimension in terms of 
BPD compressors. We then study the relationships between BPD compression, and 
the standard Lempel-Ziv (LZ) compression algorithm, and show that in contrast to 
the finite-state compressor case, LZ is not universal for bounded pushdown compres- 
sors in a strong sense: we construct a sequence that LZ fails to compress significantly, 
but that is compressed by at least a factor 2 by a BPD compressor. As a corollary 
we obtain a strong separation between finite-state and BPD dimension. 

Keywords 

Information lossless compressors, finite state (bounded pushdown) dimension, Lempel-Ziv 
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1 Introduction 



Effective versions of fractal dimension have been developed since 2000 P, [TU] and used 
for the quantitative study of complexity classes, information theory and data compression, 
and back in fractal geometry (see recent surveys in [HI [71 112] )• Here we are interested 
in information theory and data compression, where it is known that for several different 
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bounds on the computing power, effective dimensions capture what can be considered the 
inherent information content of a sequence in the corresponding setting [12]. In the today 
reahstic context of massive data streams we need to consider very low resource-bounds, 
such as finite memory or finite-time per input symbol. 

The finite state dimension of an infinite sequence [3J, is a measure of the amount of ran- 
domness contained in the sequence within a finite-memory setting. It is a robust quantity, 
that has been shown to admit several characterizations in terms of finite-state information 
lossless compressors (introduced by Huffman [S], [3]), finite-state decompressors [H [TB] . 
finite-state predictors in the logloss model P, and block entropy rates [2]. It is an effec- 
tivization of the general notion of Hausdorff dimension at the level of finite-state machines. 
Informally, the finite state dimension assigns every sequence a number s G [0, 1], that char- 
acterizes the randomness density in the sequence (or equivalently its compression ratio), 
where the larger the dimension the more randomness is contained in the sequence. 

In a recent line of research. Doty and Nichols [5] investigated a variant of finite-state 
dimension, where the finite state machine comes equipped with an infinite memory stack 
and is called a pushdown automata, yielding the notion of pushdown dimension. Hence 
the pushdown dimension of a sequence, is a measure of the density of randomness in the 
sequence as viewed by a pushdown automata. Since a finite-state automata is a special 
case of a pushdown automata, the pushdown dimension of a sequence is a lower bound 
for its finite state dimension. It was shown in [5], that there are sequences for which the 
pushdown dimension is at most half its finite state dimension, hence yielding a strong 
separation between the two notions. Unfortunately the notion of pushdown dimension is 
not known to enjoy any of the equivalent characterizations that finite state dimension does. 
Moreover, the computation time per input symbol can be unbounded, which rules out this 
model for many real-time applications. 

In this paper we introduce a variant of pushdown dimension called bounded pushdown 
(BPD) dimension: Whereas pushdown automata can choose not to read their input and 
only work with their stack for as many steps as they wish (each such step is called a 
lambda transition), we add the additional real-time constraint that the sequences of lambda 
transitions are bounded, i.e. we only allow a bounded number of stack movements per each 
input symbol. 

We define the notion of bounded pushdown dimension as the natural effectivitation of 
Hausdorff dimension via Lutz's gale characterization [H]. We provide evidence that bounded 
pushdown dimension is a robust notion by giving a compression characterization; i.e. we 
introduce BPD information-lossless compressors and show that the best compression ratio 
achievable on a sequence by BPD compressors is exactly its BPD dimension. 

In the context of compression, we study the relationship between BPD compression and 
the standard Lempel-Ziv (LZ) compression algorithm [H]. It is well known that the LZ 
compression ratio of any sequence is a lower bound for its finite state compressibility [Ti] . 
i.e. LZ compresses every sequence at least as well as any finite-state information lossless 
compressor. We show that this fails dramatically in the context of BPD compressors, by 
constructing a sequence that LZ fails to compress significantly, but is compressed by at least 
a factor 2 by a BPD compressor, thus yielding a strong separation between LZ and BPD 
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dimension. This implies that we have the same separation between LZ and (unbounded) 
pushdown dimension, and between finite state dimension [3j and BPD dimension. 

Section 2 contains the prehminaries, section 3 presents BPD dimension and its basic 
properties, section 4 proves the equivalence of BPD compression and dimension and section 
5 contains the separation of BPD compression from Lempel Ziv compression. The proofs 
are postponed to the appendix. 



2 Preliminaries 

We write Z for the set of all integers, N for the set of all nonnegative integers and Z+ for 
the set of all positive integers. Let E be a finite alphabet, with |E| > 2. E* denotes the 
set of finite strings, and the set of infinite sequences. We write for the length of 
a string w in S*. The empty string is denoted A. For S G and i,j G N, we write 
Sf^.j] for the string consisting of the i^^ through j**^ symbols of S, with the convention 
that S[i..j] = X if i > j, and ^[O] is the leftmost symbol of 5*. We write S[i] for S[i..i] (the 
i*^ symbol of S). For w G S* and 5* G we write w ^ S if w is a. prefix of S, i.e., if 
w = S[0..\w\ — 1]. All logarithms are taken in base 



3 Bounded Pushdown Dimension 

In this section we first recall Lutz's characterization of Hasudorff dimension in terms of 
gales that can be used to effectivize dimension. Then we introduce Bounded Pushdown 
dimension based on the concept of BPD gamblers and give its basic properties. 
Definition. [9j Let s G [0,cxd). 

1. An s-gale is a function d : H* ^ [0, oo) that satisfies the condition 

d{wa) 

d{w) = ^-^^^ (1) 

for all w G S*. 

2. A martingale is a 1-gale. 

Intuitively, an s-gale is a strategy for betting on the successive symbols of a sequence 
S G For each prefix w of S, d{w) is the capital (amount of money) that d has after 
having bet on S'[0..|w| — 1]. When betting on the next symbol 6 of a prefix wb of S, 
assuming symbol b is equally likely to be any value in S, equation ([T]) guarantees that the 
expected value of d{wb) is ^ d{wa) = \'E\'^~^d{w). If s = 1, this expected value is 

exactly d{w), so the payoffs are "fair". 
Definition. Let d be an s-gale, where s G [0, oo). 

1. We say that d succeeds on a sequence S G if 
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limsup (i(S'[0..n — 1]) = oo. 

n— >oo 



2. The success set of d is 

S'^[d] = {5 G I succeeds on S}. 

Observation 3.1 Let s,s' G [0, oo). For every s-gale d, the function d' : T,* ^ [0, c)o) 
defined by d'{w) = is an s'-gale. Moreover, if s < s' , then S°°[d] C S°°[d']. 

Lutz characterized Hausdorff dimension using gales as follows. 

Theorem 3.2 /.^/ Given a set X C S°°, z/dimH(X) is the Hausdorff dimension of X fS^, 
then 

dimH(X) = inf{s | there is an s — gale d such that X C 5'°°[(i]} 

The idea for a Bounded Pushdown dimension is to consider only s-gales that are com- 
putable by a Bounded Pushdown (BPD) gambler. Bounded Pushdown gamblers are finite- 
state gamblers [3] with an extra memory stack, that is used both by the transition and 
betting functions. Additionally, BPDG's are allowed to delay reading the next character 
of the input -they read A from the input- in order to alter the content of their stack, but 
they cannot do this more than a constant number of times per each input symbol. During 
such A-transitions, the gambler's capital remains unchanged. 

The betting function returns a probability measure over the input alphabet. 
Definition. Let S be a finite alphabet. Aq(S) is the set of all rational- valued probability 
measures over E, i.e., all functions n : S — > [0, 1] fl Q such that Yl ^l*^) = ^■ 

We are ready to define BPD gamblers. 
Definition. A bounded pushdown gambler (BPDG) is an 8-tuple G ={Q, S, F, 5, /?, go, 
zq, c) where 

• Q is a finite set of states, 

• S is the finite input alphabet, 

• F is the finite stack alphabet, 

• 5 : (SU{A}) xF ^ QxF* is the transition function (for simplicity we use the nota- 
tion 6{q, b,a) = -L when undefined; and we write 6{q, b, a) = {Sq^q, b, a), 6r*{q, b, a))), 

• P : Q X T ^q(S) is the betting function, 

• go G Q is the start state, 

• 2;o ^ r is the start stack symbol, 

• c G N is a constant such that the number of A-transitions per input symbol is at most 
c, 
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with the two additional restrictions: 

1. for each q & Q and a e F at least one of the following holds 

• S{q, A, a) —J- 

• 5{q, b, a) =± for all 6 e E 

2. for every q & Q, b & 12 \J {A}, either S{q, b, zq) =±, or d{q, b, zq) = {q', vzq), where q' 
e Q and V e r*. 

We denote with BPDG the set of all bounded pushdown gamblers. 

The transition function 5 outputs a new state and a string z' e T*. Informally, 
w, a) — {q', z') means that in state q, reading input w, and popping symbol a from the 
stack, S enters state q' and pushes z' to the stack. 

Note that w can be A (ic, a A-transition: the input is ignored and 6 only computes with 
the stack) but this only happens at most c times per input symbol. Any pair (state, stack 
symbol) can either be a A-transition pair or a non A-transition pair exclusively, because 
the first additional restriction enforces determinism. 

Moreover, since Zq represents the bottom of the stack, we restrict 5 so that Zq cannot 
be removed from the bottom by the second additional restriction. 

We can extend 6 in the usual way to 



(5* : Q X (E U {A}) X r 



g xr. 



where for aR q e Q, a e F, v e T*, and 6 e E U {A} 





otherwise. 




where 



S\q,X,v) = S{q,X,v). 
Since S is c-bounded we have that for any q e Q, v E T*, 

S^+\q,X,v) = ± 



We also consider the extended transition function 



5** : Q X E* X r 



Qxr, 



defined for ell q e Q, a e T, v e T*, w e E*, and 6 e E by 



S**{q, A, av) — {q, av) 
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S**{q, wb, av) = 6{6Q{q, A, av), b, 6^, (g, A, av)) 

if 6**{q,w,av) = {q,av), 6'^{q,X,av) and 6^~^^{q, X,av) =±, i < c. 

That is, A-transitions are inside the definition of 6**{q,b,av), for 6 G S. Notice that 
6** is not defined on an empty stack string, therefore av needs to be long enough in order 
that 6**{q,b,av) 7^±. 

We denote 6** by 6, and S{qQ, w, zq) by 6{w). We write 6 = {6q, 6r*) for simphcity. 

We also consider the usual extension of P 

/?* : Q X r+ ^ Aq(S), 
defined for all q E Q, a E T, and f G F* by 

/3*{q,av) = f3{q,a), 

and denote P* by p. 

We use BPDG to compute martingales. Intuitively, suppose a BPDG G is to bet on 
sequence S has already bet on w \Z S, with current capital x G Q, current state q E Q and 
current top stack symbol a. Then for 6 G E, G bets the quantity a)(b) of its capital 
that the next symbol of S is b. If the bet is correct (that is, if wb C S) and since payoffs 
are fair, G has capital |S|x/5(g, a)(&). Formally, 

Definition. Let G = {Q,T,,T,6, P,qo, zo,c) be a bounded pushdown gambler. The 
martingale of G is the function 

: S* [0, cx)) 

defined by the recursion 

daiX) = 1 
daiwb) = mdG{w)Pi5iwmb) 

for all U7 G S* and 6 G S. 

By Observation 13.11 a BPDG G actually yields an s-gale for every s G [0, oo). We call 
it the s-gale of G, and denote it by 

A bounded pushdown s-gale is an s-gale d for which there exists a BPDG such that d^ = d. 

The first two properties of BPD gamblers are that any number of A-transitions can 
be replaced by a single A-transition and that the stack alphabet does not give additional 
power. 

Proposition 3.3 Let G = (Q, S, F, 5, /5, go, ^o, c) be a BPDG. Then there is a BPDG G' = 
{Q' , S, F', 5', ^Q, -Zq, 1) such that do = dc ■ 

From now on we shall assume that the maximum number of A-transitions c is 1. 

Proposition 3.4 Let G = {Q, S, F, S, /?, go, zq, c) be a BPDG. Then there is a BPDG G' = 
{Q', S, {0, 1, Zq}, 6', P', go, Zq, c') such that do = dc' ■ 
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Let us define bounded pushdown dimension. Intuitively, the BPD dimension of a se- 
quence is the smallest s such that there is a BPD-s-gale that succeeds on the sequence. 
Definition. The bounded pushdown dimension of a set X C E°° is 

dimBPD(^) = inf{s | there is a bounded pushdown s — gale d such that X C 

4 Dimension and compression 

In this section we characterize the bounded pushdown dimension of individual sequences 
in terms of bounded pushdown compressibility, therefore BPD dimension is a natural and 
robust definition. 

Definition. A bounded pushdown compressor (BPDC) is an 8-tuple 

C = {Q, E, r, 5, 1/, go, ^0, c) 

where 

• Q is a finite set of states, 

• E is the finite input and output alphabet, 

• r is the finite stack alphabet, 

• 5 : (5 X (E U {A}) X r ^ (5 X r* is the transition function, 

• i/:(5xExr^E*is the output function, 

• qo & Q is the initial state, 

• e r is the start stack symbol, 

• c e N is a constant such that the number of A-transitions per input symbol is at most 
c, 

with the two additional restrictions: 

1. for each g G Q and a e F at least one of the following holds 

• S{q, A, a) =± 

• S{q, b, a) —± for all 6 e E 

2. for every q & Q, b & U {A}, either 5{q, b, zq) —-i-, or S{q, b, zq) = (g', vzq), where q' 
e Q and V e r*. 
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We extend S to S** : Q x E* x r+ ^ Q x T* as before, and denote S** by S and S{qo, w, Zq) 
by 5{w). 

For q E Q, w E Ti* and z G T~^, we define the output from state g on input w reading 
z on the top of the stack to be the string i'*{q, w, z) (denoted by w, z)) with 

i^(q,X,z) = A 

u(q, wb, z) = v{q, w, z)u{5Q(q, w, z), b, 5r*(g, w, z)) 
for w e T,* and 6 e E. We then define the output of C on input w e T,* to be the string 

C{w) = iy{qo,w,Zo). 

We can restrict A-transitions to a single one and the stack alphabet to three symbols. 

Proposition 4.1 Let C = {Q, S, T, S, u, qo, zq, c) be a BPDC. Then there is a BPDC C = 
(Q', E, r', 5\ i/', g^, 4, 1) such that C{w) = C'{w) for every w e E*. 

Proposition 4.2 Let C = (Q, E, T, 6, u, qo, zq, c) be a BPDC. Then there is a BPDC C = 
(Q', S, {0, 1, z'q\, S', u', q'o, Zq, c') such that C{w) — C'{w) for every w e E*. 

We are interested in information lossless compressors, that is, w must be recoverable 

from C{w) and the final state. 

Definition. A BPDC C — {Q, E, F, S, v, qo, zq) is information-lossless {IL) if the function 

E* ^ E* X g 

w^(C(w),5q(w)) 

is one-to-one. An information-lossless bounded pushdown compressor (ILBPDC) is a 
BPDC that is IL. 

Intuitively, a BPDC compresses a string w if |C('U7)| is significantly less than \w\. Of 
course, if C is IL, then not all strings can be compressed. Our interest here is in the 
degree (if any) to which the prefixes of a given sequence S e E°° can be compressed by an 
ILBPDC. 

Definition. If C is a BPDC and S e E°°, then the compression ratio of C on 5" is 

n— >oo ri 

The BPD compression ratio of a sequence is the best compression ratio achievable by 
an ILBPDC, that is 

Definition. The bounded pushdown compression ratio of a sequence S e E°° is 

Pbpd(^) = inf{pc(^) I C is a ILBPDC}. 

The main result in this section states that the BPD dimension of a sequence and its 
ILBPD compression ratio are the same, therefore BPD dimension is the natural concept 
of density of information in the BPD setting. 
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Theorem 4.3 For all S 

dimBPD(5') = Pbpd(5'). 

5 Separating LZ from BPD 

In this section we prove that BPD compression can be much better than the compression 
attained with the celebrated Lempel-Ziv algorithm. 

We start with a brief description of the LZ algorithm [H] . 

We finish relating BPD dimension (and compression) with the Lempel-Ziv algorithm. 
Given an input a: G S*, LZ parses x in different phrases 

such that every prefix y □ Xj, appears before Xi in the parsing (i.e. there exists j < i s.t. 
Xj = y). Therefore for every = xi(^i)hi for < i and bt G S. We sometimes denote 
the number of phrases in the parsing of x as C{x). 

LZ encodes by a prefix free encoding of and the symbol 6,, that is, if x = 
X\X2 . . . Xyi as before, the output of LZ on input x is 

LZ{x) = Q(i)6iQ(2)&2 • • • Cl(n)bn 

where q is a prefix-free coding of i (and Xq = A). 

LZ is usually restricted to the binary alphabet, but the description above is valid for 
any S. 

For a sequence S G the LZ compression ratio is given by 

. ^ \LZiS[0...n-l])\ 
pLz[S) =limmf . 

n^oo TL 

It is well known that LZ [14j yields a lower bound on the finite-state dimension (or finite- 
state compressibility) of a sequence [Tl|, ie, LZ is universal for finite-state compressors. 

The following result shows that this is not true for BPD (hence PD) dimension, in a 
strong sense: we construct a sequence S that cannot be compressed by LZ, but that has 
BPD compression ratio less than |. 

Theorem 5.1 For every m G N, there is a sequence S G {0, 1}°° such that 

pLz{S) > 1 - — 
m 

and ^ 

dimBPD(5') < -. 

As a corollary we obtain a separation of finite-state dimension and bounded pushdown 
dimension. A similar result between finite-state dimension and pushdown dimension was 
proved in [3]. 
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Corollary 5.2 For any m G there exists a sequence S G {0, 1}°° such that 

dimFs(S') > 1 

m 

and ^ 

dimBPD(5') < -. 

Conclusion 

We have introduced Bounded Pushdown dimension, characterized it with compression and 
compared it with Lempel-Ziv compression. It is open if there is a BPD compressor that 
is universal for Finite-State compressors, which is true for the Lempel-Ziv algorithm, and 
whether Lempel-Ziv compression can surpass BPD-compression for some sequence. 

References 

[1] K. B. Athreya, J. M. Hitchcock, J. H. Lutz, and E. Mayordomo. Effective strong 
dimension in algorithmic information and computational complexity. SIAM Journal 
on Computing. To appear. 

[2] Chris Bourke, John M. Hitchcock, and N. V. Vinodchandran. Entropy rates and 
finite-state dimension. Theor. Comput. Sci., 349(3) :392-406, 2005. 

[3] Jack J. Dai, James 1. Lathrop, Jack H. Lutz, and Elvira Mayordomo. Finite-state 
dimension. Theoretical Computer Science, 3 10 (1-3): 1-33, January 2004. 

[4] D. Doty and P. Moser. Personal communication, based on [13]. 2006. 

[5] David Doty and Jared Nichols. Pushdown dimension. Theoretical Computer Science. 
To appear. 

[6] K. Falconer. The Ceometry of Fractal Sets. Cambridge University Press, 1985. 

[7] J. M. Hitchcock, J. H. Lutz, and E. Mayordomo. The fractal geometry of complexity 
classes. SICACT News Complexity Theory Column, 36:24-38, 2005. 

[8] D. A. Huffman. Canonical forms for information-lossless finite-state logical machines. 
Trans. Circuit Theory, pages 41-59, 1959. 

[9] J. H. Lutz. Dimension in complexity classes. SIAM Journal on Computing, 32:1236- 
1259, 2003. 

[10] J. H. Lutz. The dimensions of individual strings and sequences. Information and 
Computation, 187:49-79, 2003. 



10 



[11] J. H. Lutz. Effective fractal dimensions. Mathematical Logic Quarterly, 51:62-72, 
2005. 

[12] E. Mayordomo. Effective fractal dimension in algorithmic information theory. In New 
Computational Paradigms: Changing Conceptions of What is Computable. Springer- 
Verlag, 2007. To appear. 

[13] D. Sheinwald, A. Lempel, and J. Ziv. On compression with two-way head machines. 
In Data Compression Conference, pages 218-227, 1991. 

[14] Jacob Ziv and Abraham Lempel. Compression of individual sequences via variable- 
rate coding. IEEE Transactions on Information Theory, 24(5):530-536, 1978. 



11 



Technical Appendix 



This appendix is devoted to proving Theorem 14.31 and Theorem 15.11 For the first one, 
we need the following: 



A Proof of Theorem 4.3 



Definition. A BPDG G = {Q,Tj,r,6, (3,qo, Zq) is nonvanishing if < f3{q,z){b) < 1 for 
all q E Q, b E T, and z E T. 

Lemma A.l For every BPDG G and each e > 0, there is a nonvanishing BPDG G' such 
that for allw G S*, c/g'H > iSl-^l^-lrfcH- 

Proof of Lemma EUl . Let G = (Q, S, 6, p, go, T, zq) be a BPDG, and let e > 0. For 
each q E Q, z E T, b E T,, 

l-|Sr^5^/3(g,^)(6) = l-|S|-^>0, 

so we can fix a rational (3'{q, z){b) such that 

|S|-^/5(g,z)(6)</3'(g,z)(6)<l-|Sr^ J2 /5(g,^)(a) 

and 

Y,/3'iq,z)ib) = l. 

Then, < (3'{q,z){b) < 1 for each g G Q, 6 G S and ^ G F, therefore the BPDG G' = 
{Qy'Ejd, P' ,qo,T, Zq) is nonvanishing. 
Also, for all q E Q , b E T,, z E T , 

f3'{q,z){b)>\J:rf3{q,z){b) 

so for all w E E*, ddw) > 

□ 

Proof of Theorem Let S E For each n E N, let = S[0..n - 1]. 

To see that dimBPD('S') < Pbpd(5'), let s > s' > Pbpd{S). It suffices to show that 
dimBPD(5') < s. By our choice of s', there is an 1-ILBPDC G = {Q,'E,T,6,i',qo, zq) for 
which the set 

/ = {n G N I \G{wn)\ < s'n} 

is infinite. 
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CONSTRUCTION A.l Given a 1 -bounded pushdown compressor (BPDC) 

C = {Q,T,,T,6,h',qQ,zo), and /c G Z+ , we construct the 1-bounded pushdown gambler 

(BPDG) a = G{C, k) = (g', S, r', 6\ I3\ q'q, z'q) as follows: 

Q' = Q X {0,l,...,k-l} 
Qo = (^0, 0) 

4fc-l 
i=2k 

iv) z'q = zf 

v) y{q,i) eQ',be S,aGr', 

S'{{q,i), b,a) = ( (6Q{q,b,a), [i + 1) mod k) ,Sr*{q,b,a) 



where for each z G (r')"*", 2; G r+ the Y -string obtained by concatenating the symbols of 
z, and for each y G F"*^, if y = ^1^2 ■ ■ 'y2ki+n with n < 2k, then y G (r')"*" is such that 

Hi = Ul' ' ' y2k+n, 1/2 = 2/2fc+n+l ' ' ' UAk+ri, ■ ■ ■ , Vl = 2/2fc(i-l)+n+l ' ' ' y2kl+n- 

vi) y{q,t) eQ',aeT',beJ: 

a{q,b^''-'-\a) 



(3'{{q,t),a){b) 



where a {q, A, a) = XI 1^1 



-\i/{q,x,a)\ 



Lemma A. 2 In Gonstruction \A.li if \w\ is a multiple of k and u G H-^, then 

Proof of Lemma IA.21 We use induction on the string u. If m = A, the lemma is 
clear. Assume that it holds for u, where u G T^^^, and let 6 G S. Then 

, f .^ 1^1 ^(^q(wm), 6r4wu)) 
dciwub) = |S| z=^^ dG{wu) 

^1^1 hk5q(-) A<5r* M)\ cT{6Q{wub), 6r^ub)) ^^^^^^^^^ 

a{6Q{wu), S^-l«l,5r*(tfM)) 

so by the induction hypothesis the lemma holds for ub. 
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□ 

Lemma A. 3 In Construction \A.R if w = woWi ■ ■ -Wn-i, where each Wi E 11^ , then 

\^\\w\-\C(w)\ 



dciw) 



n-l 

n (^{Sq^Wq ■ ■ ■ Wi-i), E'^, 5r' (wo ■ ■ ■ Wi^i)) 

1=0 



Proof of Lemma IA.31 We use induction on n. For n = 0, the identity is clear. 
Assume that it holds for w = WqWi ■ ■ ■ Wn-i, with each Wi G S'^, and let w' = WqWi ■ ■ ■ Wn- 
Then Lemma IA.2I with u = Wn tells us that 

|^|fc-|i/((5Q(ui),w„,(5p* (ui))| 

(T((5Q(u7),S^,(5r.(u;)) 
whence the identity holds for w' by the induction hypothesis. 

□ 

Lemma A. 4 In Construction \A.R if C is IL and \w\ is a multiple of k, then 

where I = [log \Q\] and m = max{|z/(g, b,a)\ \ q E Q,b E I],a E F^}. 
Proof of Lemma IA.4L We prove that for each z E T,*, 

To see this, fix z G S* and observe that at most \Q\ strings w E T,^ can have the same 
output from state Sq{z) with stack content Sr*{z). Therefore, the number of w G for 
which \i'{6Q{z),w,6r*{z))\ = j does not exceed Hence 

mk 

ai6Q{z),^\S^)) = ^-^■'('^(^^'^''rHz))] < ^|Q||sp|S|-^' = \Q\{mk + l) 
It follows by Lemma IA.3I that 



□ 



Lemma A. 5 In Construction \A.R if C is IL, then for all w E T,* , 

dciw) > |^||u'|-|C(u))| — ^(i+logm+logfe+l) — (fcm+i+logm+logfc+l) 

where I = [log IQl] and m = max b,a)\ \ q E Q,b E Ti,a E F^}. 
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Proof of Lemma IA.51 Assume the hypothesis, let / and m be as given, and let w G 
S*. Fix < j < k such that \w\ + j is divisible by k. By Lemma [A.4I we have 

daiw) > m-^dciwO^) 

> \Yj\-j+\w(P\-\CiwO^)\-^^{l+logm+logk+l) 

— I S I ' ^ I ) I - ^ C+log m+log fc+1)- 1 (i+log m+log k+1) 

^ I S I I^T^ "i+log k+l) — (km+l+log m+log A;+l) 

□ 

Let / = [loglQII and m = max{|z/(g, 6, a)| | g G Q,6 G S,a G F^}, and fix A; G Z"*" such 
that '+'°g"^+'°gfc+i < s - s'. Let G = G{C, k) be as in Constructional Then, by Lemma 
IA.5t for all n G / we have 

G \ ^/ — I I 

> I X: I (^-^'^ '"^'"'^ ""V"^ ''^^ )ra-(fcm+;+log m+log fc+1) 

Since s - s' - > o, this implies that S G 

Thus, dimBPD(5') < s. 

To see that Pbpd(5') < dimBPD(5'), let s > s' > s" > dimBPD(5'). It suffices to show 
that Pbpd('S') < s. By our choice of s", there is a 1-BPDG G such that the set 

J= {n G N I d'aiwn) > 1} 

is infinite. By Lemma [A. II there is a nonvanishing 1-BPDG G such that 
d^iw) > |S|('^"-^')I"'Ic/g(«^) for all w G S*. 

CONSTRUCTION A.2 Let G = (Q, S, F, 5, /?, go, ^o) be a nonvanishing 1-BPDG, and 
let k E Z'^ . For each 2; G F* (long enough for dc^^^iw) to be defined for all w G E'^j 
and q E Q, let Gq^z = S, F, ^, /3, g, z), and define Pq^z '■ [0,1] by Pq^ziw) = 

iT^l'^^dc^ ^{w) . Since G is nonvanishing and each dc^^ is a martingale with ^^^^(A) = 1, 
each of the functions Pg^z is a positive probability measure on "E^ . For each z E T* , q E Q , let 
Qq z : S'^ — i> S* he the Shannon-Fano-Elias code given by the probability measure Pg^z- Then 

Iq Aw) = l+\l0g^] 

for all q E Q and w E S'^, and each of the sets range{QqA is an instantaneous code. We 
define the 1-BPDC G = G{G, k) = {Q', S, F', 6', z/', q^, Zq) whose components are as follows: 

i) Q' = Qx S<'= 
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4fc-l 

Hi) r = [j r 

i=2k 

iv) z'q = zf 

v) y{q,w) eQ',heT.,ae V, 



5'{{q,w),b,a) 



(g, wb, a) if \w\ < k — 1, 

{6Q{q,wb,a), X,6r*{q,wb,a)) if \w\ = k - 1. 

vi) \/{q,w) eQ',bei:, ae V, 



u'{{q,w),b, a) 



X if \w\ < k — 1, 

Qq,a{wb) if \w\ = k - 1. 



Since each range(0g_2) is an instantaneous code, it is easy to see that the BPDC C 
C{G, k) is IL. 

Lemma A. 6 In Construction \A.^ if \w\ is a multiple ofk, then 

2^ 



\C{w)\ < {l + ^\w\-\ogdG{w) 



Proof of Lemma IA.61 Let w = WqWi ■ ■ ■ Wn-i, where each Wi G S'^. For each 
< i < n, let qi = Sq{wo ■ ■ ■ and Zi = Sr*{wo ■ ■ ■ Then, 



n-l 
1=0 

n— 1 ^ n— 1 



1=0 

n-l , X n-l 



i=0 ^ "-^J.ji.s / i=o 

2 

{k + 2)n - logrfclw) = (1 + ~ logrfG(u^) 



□ 



Lemma A. 7 In Construction \A.Si for all w G E*, 

2^ 



\C{w)\<(l + -)\w\-\ogdG{w). 



Proof of Lemma I A. 71 If \w\ is multiple of k, then we apply the Lemma [A. 61 
Otherwise, let w = w'z, where \w'\ is a multiple of k and \z\ = j , < j < k. 

Then, Lemma IA.6I tell us that 

\C{w)\ = \C{w')\ 

< (^l + ^^\w'\-\0gdG{w') 

2- 



= (^1 + -iogrfG(w) - ^ 



□ 



Fix k > jz^, and let C = C{G,k) be as in Construction IA.2i Then Lemma [A. 71 tell us 
that for all n & J, 



C{wn) \ < (1 + j^^n- log d^iwn) 

< ^1 + ^ + s' - s"^n - logdciwr, 

< + s'^n - log d'aiwn) 



<{l + s']n 

< sn. 



Thus, Pbpd(5') < s. 



□ 



B Proof of Theorem 5.1 



For a string x, denotes x written in reverse order. 

Proof of Theorem 15.11 Let m G N, and let k = k{m) be an integer to be determined 
later. For any integer n, let T„ denote the set of strings x of size n such that V does not 
appear in x, for every j > k. Since T„ contains {0, l}'^"^ x {0} x {0, l}'^^^ x {0} . . . (i.e. 
the set of strings whose every kth bit is zero), it follows that |T„| > 2""', where a = 1 — 1/k. 

Remark B.l For every string x E Tn there is a string y G T„-i and a bit b such that 
yb = X. 
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Let An = {oi, . . .a„} be the set of palindromes in T„. Since fixing the n/2 first bits 
of a pahndrome (wlog n is even) completely determines it, it follows that < 22. 
Let us separate the remaining strings in T„ — An into two sets X„ = {xi,...Xt} and 

= {yi, ■ ■ ■ Vt} with {xi)~^ = Hi for every 1 < i <t. Let us choose X, Y such that xi and 
yt start with a zero. We construct S in stages. For n < k — 1, Sn is an enumeration of all 
strings of size n in lexicographical order. For n > k, 

5„ = ai . . . a„ 1^" xi . . . xt 1^"+^ yt---yi 

i.e. a concatenation of all strings in An (the A zone of Sn) followed by a flag of 2n 
ones, followed by the concatenations of all strings in X (the X-zone) and Y (the Y zone) 
separated by a flag of 2r2 + 1 ones. Let 

O — D1D2 . . . Ok~l i i . . . i DkOk+1 ■ ■ ■ 

i.e. the concatenation of the S/s with some extra flags between Sk-i and Sk- We claim 
that the parsing of Sn {n > k) by LZ, is as follows: 

Sn = ai, . . . ,au, 1^", xi,...,xt, 1^"+^ 2/4,..., 2/1. 

Indeed after 6*1, .. . Sfc-i 1*^ l'^"*"^ . . . LZ has parsed every string of size < k — 1 and 

the flags 1'^ l'^"^^ . . . Together with Remark [B. 11 this guarantees that LZ parses Sn 

into phrases that are exactly all the strings in T„ and the two flags 1^", l^^+i. 

Let us compute the compression ratio Plz{S). Let n,i be integers. By construction of 
S, LZ encodes every phrase in Si (except the two flags), by a phrase in Si^i (plus a bit). 
Indexing a phrase in Si-i requires a codeword of length at least logarithmic in the number 
of phrase parsed before, i.e. log(C(S'iS'2 . . . Si-2))- Since C{Si) > |Tj| > 2"*, it follows 

90(1— 1) na 

C{S, . . . S,^2) > Yl 2"'' = ^ ^2^^^^"'^ 

j=i 

where b = b{a) is arbitrarily close to L Letting ti = |Tj|, the number of bits output by LZ 
on Si is at least 

C{S,) logC(5i . . . 5,_2) > t.log62'^(^-i) 

> ctiit - 1) 

where c = c{b) is arbitrarily close to 1. Therefore 

n 

\LZ{S,...Sn)\>J2^tj(3-l) 

i=i 

Since IS*! . . . 5'„| < 2fc^ + YTj=iU'^3 + ^j), (the two flags plus the extra flags between Sk-i 
and Sk) the compression ratio is given by 

i^7(j-i) 

_ 2fc^+E;=i(t.+4j) 

' '2F + E"=ij(t.+4) 
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The second term in Equation [H] can be made arbitrarily small for n large enough: Let 



M < n, we have 



n M n 

2k' + Y,j{t, +A) >2k' + Y,jt, + {M +1) tj 

j=l j=l j=M+l 

M n n 

= 2e + Y,jt,+M J2 E 

j=l j=M+l j=M+l 

M n n 

>2e + Y,3h + M h+ E 2'^' 

3=1 j=M+l j=M+l 

M n 

j=l j=M+l 
n M 

>M Y + M{2k'' + 2n{n + 1) + ^ ^i) ^ enough 

j=M+l j=l 



Hence 



pLz{Si...Sn) > C - — 



which by definition of c, M can be made arbitrarily close to 1 by choosing k accordingly, 
i.e 



pLz{Si ■■■Sn)>l-—. 

m 



Let us show that dimBPD(5') < \. Consider the following BPD martingale d. Informally, 
d on Sn goes through the An zone until the first fiag, then starts pushing the whole X 
zone onto its stack until it hits the second fiag. It then uses the stack to bet correctly on 
the whole Y zone. Since the Y zone is exactly the X zone written in reverse order, d is 
able to double its capital on every bit of the Y zone. On the other zones, d does not bet. 
Before giving a detailed construction of (i, let us compute the upper bound it yields on 
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dimBPD('S'). 

log d{S,...S^) 



dimBPD('S') < 1 — limsup 

n— >oo 

< 1 — lim sup 



jS*! . . . 

IF- 1 



2fc^+E;=i(ji^.i+4j) 

< 1 — lim sup 



En ■ \Tj\ — \ Aj I 
j=l3 2 



Since 



^11,. 2p+e;=i(ji^.i+4j) 

< — — limsup — — — , ,^ , -. 



2fe^ + E?=i(j|^.l+4j) ^, E;=iJ(I^.I +4 + 2^2) 
lim sup — r — — — < lim sup 



< limsup 



E;=,j(2* + 2i) 



E-=i2"^' 



. 3n 



, V ^2- 
< lim sup 

— ^ nan 

= 0. 

It follows that ^ 

dimBPD('S') < -. 

Let us give a detailed description of d. Let Q be the following set of states: 

• The start state qo, and qi, . . .q^, the "early" states that will count up to 

\SiS2...Sk-i 1*^+^ ...i^*'-^!. 

• ^Q, . . . , the A zone states that cruise through the A zone until the first flag. 

• q^^ the first flag state. 

• 1q 1 ■ ■ ■ )Q.k zone states that cruise through the X zone, pushing every bit on 
the stack, until the second flag is met. 

• qQ, . . . ,ql which after the second flag is detected, pop k symbols from the stack that 
were erroneously pushed while reading the second flag. 

• q'f the second flag state. 

• q^ the betting on zone Y state. 
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Let us describe the transition function S : Q x {0,1} x {0,1} Q x {0,1}. First S counts 
until V i.e. ior i — 0, . . .v — 1 

SiQi: X, y) = {qi+i,y) for any x, y 
and after reading v bits, it enters in the first A zone state, i.e. for any x, y 

6{qy,x,y) = {qo,y). 

Then S skips through A until the string l'' is met, i.e. ior i — 0, . . .k — 1 and any x, y 



S{qt,x,y) 

and 



{qf+i,y) iix^l 
{qly) ifx = 



Kqlx,y) = {q'f,y). 

Once \^ has been seen, 5 knows the first flag has started, so it skips through the flag until 
a zero is met, i.e. for every y 

S{q^^,x,y) - 



{q^f,y) ifx = l 
{q^,Oy) iix^O 



where state qQ means that the first bit of the X zone (a zero bit) has been read, therefore 
6 pushes a zero. In the X zone, delta pushes every bit it sees until it reads a sequence of 
k ones, i.e until the start of the second flag, i.e for i = 0, . . . /c — 1 and any x, y 



S{qf,x,y) 

and 



{qf+i,xy) iix = l 
{qo,xy) ifx^O 



KQ.k^x,y) = {ql,y). 

At this point, 5 has pushed all the X zone on the stack, followed by k ones. The next step 
is to pop k ones, i.e for i = 0, ... A; — 1 and any x, y 

S{ql,x,y) = A) 

and 

S{ql,x,y) = {ql^,y). 

At this stage, 5 is still in the second flag (the second flag is always bigger than 2k) therefore 
it keeps on reading ones until a zero (the first bit of the Y zone) is met. For any x, y 



5{,q^^,x,y) 



{q^^.y) ifx=l 
{q\ A) if X = 0. 
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On the last step 5 has read the first bit of the Y zone, therefore it pops it. At this stage, 
the stack exactly contains the Y zone (i.e. the X zone written in reverse order) except 
the first bit; S thus uses its stack to bet and double its capital on every bit in the Y zone. 
Once the stack is empty, a new A zone begins. Thus, for any x, y 

5{q'',x,y)^{q',X). 

and 

r/ b ^ /(9?>^o) if a; = 1 
S{q ,x,Zo) = < 

[{qo,Zo) ifa; = 0. 

The betting function is equal to 1/2 everywhere (i.e no bet) except on state q'^, where 



1 ii y — z 
iiy^z. 



and j3 stops betting once start stack symbol is met, i.e. 

1 



/i^(9^;^o) = 2■ 



□ 
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